Bonus, Disclosure, and Choice: What Motivates the Creation of High-Quality Paid Reviews?

The emergence of online crowdsourcing sites has opened up new channels for third parties and companies to solicit paid reviews from people. In this paper, we investigate 1) how the introduction of monetary payments affects review quality, and 2) the impact of bonus rewards, sponsorship disclosure, and choice freedom on the quality of paid reviews. We conduct a 2×2×2 between-subjects experiment on Amazon Mechanical Turk. Our results indicate that there are no significant quality differences between paid and unpaid reviews. The quality of paid reviews improves by both the presence of additional performance-contingent rewards and the requirement to add disclosure text about material connections, and deteriorates by the restrictions imposed on the product set to be reviewed. These results have implications for websites and companies who are seeking legitimate reviews for their online products from paid workers.